High Dimensional Smoothing Based on Multilevel Analysis

A fundamental issue in Data Mining is the development of algorithms to extract useful information from very large databases. One important technique is to estimate a smooth function approximating the data. Such an approximation can for example be used for visualisation, prediction, or classification purposes. However, the number of observations can be of the order of millions and there may be hundreds of variables recorded so one has to deal with the so-called ”curse of dimensionality”. The algorithmic complexity of this process is typically of the order γ3d−2 where γ is the number of grid points in each dimension and d is the number of dimensions. We propose a method for approximating a high dimensional surface by computing a projection onto multi-level spaces of low density and we demonstrate that the algorithmic complexity of this method is proportional to (jd−1(2j+1 − 1))3, where j = ⌊log2 γ⌋ — a substantial reduction in computational work. In addition, we show that the approximation error is proportional to (j+d−1 d−1 ) 2−2j , with the proportionality constant depending on the smoothness of the computed surface.